In this problem we'll aim to get the py.test testing framework up and running in the code repository you set up in the last set of problems. We can then use it to collect and run tests of the code.
Of course py.test must actually be installed before you can use it. The commands below should work for the Anaconda Python Distribution, but if you have some other Python installation you'll want to install pytest (and its coverage plugin) as directed in the install instructions for py.test.
In [ ]:
!conda install pytest pytest-cov
Depending on what your code actually does, you might need to modify it to actually perform something testable. For example, if all it does is print something, you might find it difficult to write an effective unit test. Try adding a function that actually performs some operation and returns something different depending on various inputs. That tends to be the easiest function to unit-test: one with a clear "right" answer in certain situations.
Also be sure you have cded to the root of the repo for pytest to operate correctly.
The test must be part of the package and follow the convention that the file and the function begin with test to get picked up by the test collection machinery. Inside the test function, you'll need some code that fails if the test condition fails. The easiest way to do this is with an assert statement, which raises an error if its first argument is False.
Hint: remember that to be a valid python package, a directory must have an __init__.py
In [ ]:
!mkdir #complete
!touch #complete
In [ ]:
%%file <yourpackage>/tests/test_something.py
def test_something_func():
assert #complete
While this is not how you'd ordinarily run the tests, it's instructive to first try to execute the test directly, without using any fancy test framework. If your test function just runs, all is good. If you get an exception, the test failed (which in this case might be good).
Hint: you may need to use reload or just re-start your notebook kernel to get the cell below to recognize the changes.
In [ ]:
from <yourpackage>.tests import test_something
test_something.test_something_func()
py.testOnce you have an example test, you can try invoking py.test, which is how you should run the tests in the future. This should yield a report that shows a dot for each test. If all you see are dots, the tests ran sucessfully. But if there's a failure, you'll see the error, and the traceback showing where the error happened.
In [ ]:
!py.test
(Modify your test to fail if it succeeded before, or vice versa)
In [ ]:
!py.test
In [ ]:
!py.test --cov=<yourproject> tests/ #complete
This should yield a report, which you can use to decide if you need to add more tests to acheive complete coverage. Check out the command line arguments to see if you can get a more detailed line-by-line report.
The sub-problems below each contain different unit testing complications. Place the code from the snippets in your repository (either using an editor or the %%file trick), and write tests to ensure the correctness of the functions. Try to achieve 100% coverage for all of them (especially to catch some hidden bugs!).
Also, note that some of these examples are not really practical - that is, you wouldn't want to do this in real code because there's better ways to do it. But because of that, they are good examples of where something can go subtly wrong... and therefore where you want to make tests!
In [ ]:
#%%file <yourproject>/<filename>.py #complete, or just use your editor
# `math` here is for *scalar* math... normally you'd use numpy but this makes it a bit simpler to debug
import math
inf = float('inf') # this is a quick-and-easy way to get the "infinity" value
def function_a(angle=180):
anglerad = math.radians(angle)
return math.sin(anglerad/2)/math.sin(anglerad)
In [ ]:
#%%file <yourproject>/<filename>.py #complete, or just use your editor
def function_b(value):
if value < 0:
return value - 1
else:
value2 = subfunction_b(value + 1)
return value + value2
def subfunction_b(inp):
vals_to_accum = []
for i in range(10):
vals_to_accum.append(inp ** (i/10))
if vals_to_accum[-1] > 2:
vals.append(100)
# really you would use numpy to do this kind of number-crunching... but we're doing this for the sake of example right now
return sum(vals_to_accum)
There are (at least) two significant bugs in this code (one fairly apparent, one much more subtle). Try to catch them both, and write a regression test that covers those cases once you've found them.
One note about this function: in real code you're probably better off just using the Angle object from astropy.coordinates. But this example demonstrates one of the reasons why that was created, as it's very easy to write a buggy version of this code.
Hint: you might find it useful to use astropy.coordinates.Angle to create test cases...
In [ ]:
#%%file <yourproject>/<filename>.py #complete, or just use your editor
import math
# know that to not have to worry about this, you should just use `astropy.coordinates`.
def angle_to_sexigesimal(angle_in_degrees, decimals=3):
"""
Convert the given angle to a sexigesimal string of hours of RA.
Parameters
----------
angle_in_degrees : float
A scalar angle, expressed in degrees
Returns
-------
hms_str : str
The sexigesimal string giving the hours, minutes, and seconds of RA for the given `angle_in_degrees`
"""
if math.floor(decimals) != decimals:
raise ValueError('decimals should be an integer!')
hours_num = angle_in_degrees*24/180
hours = math.floor(hours_num)
min_num = (hours_num - hours)*60
minutes = math.floor(min_num)
seconds = (min_num - minutes)*60
format_string = '{}:{}:{:.' + str(decimals) + 'f}'
return format_string.format(hours, minutes, seconds)
Hint: numpy has some useful functions in numpy.testing for comparing arrays.
In [ ]:
#%%file <yourproject>/<filename>.py #complete, or just use your editor
import numpy as np
def function_d(array1=np.arange(10)*2, array2=np.arange(10), operation='-'):
"""
Makes a matrix where the [i,j]th element is array1[i] <operation> array2[j]
"""
if operation == '+':
return array1[:, np.newaxis] + array2
elif operation == '-':
return array1[:, np.newaxis] - array2
elif operation == '*':
return array1[:, np.newaxis] * array2
elif operation == '/':
return array1[:, np.newaxis] / array2
else:
raise ValueError('Unrecognized operation "{}"'.format(operation))
Now that you have a testing suite set up, you can try to turn on a continuous integration service to constantly check that any update you might send doesn't create a bug. We will the Travis-CI service for this purpose, as it has one of the lowest barriers to entry from Github.
In [ ]:
!py.test
This turns out to be quite convenient. If you go to the Travis web site, you'll see a "Sign in with GitHub" button. You'll need to authorize Travis, but once you've done so it will automatically log you in and know which repositories are yours.
In [ ]:
%%file .travis.yml
language: python
python:
- "3.6"
# command to install dependencies
#install: "pip install numpy" #uncomment this if your code depends on numpy or similar
# command to run tests
script: pytest
Be sure to commit and push this to github before proceeding:
In [ ]:
!git #complete
You can now click on your profile picture in the upper-right and choose "accounts". You should see your repo listed there, presumably with a grey X next to it. Click on the X, which should slide the button over and therefore activate travis on that repository. Once you've done this, you should be able to click on the name of the reposository in the travis accounts dashboard, popping up a window showing the build already in progress (if not, just be a bit patient).
Wait for the tests to complete. If all is good you should see a green check next to the repository name. Otherwise you'll need to go in and fix it and the tests will automatically trigger when you send a new update.
Push that change up and go look at travis. It should automatically run the tests and result in them failing.
Challenge your nieghbor to find the bug and fix it. Have them follow the Pull Request workflow, but do not merge the PR until Travis' tests have finished (they should run automatically, and leave note in the github PR page to that effect). Once the tests have finished, they will tell you if the fix really does cure the bug. If it does, merge it and say thank you. If it doesn't, ask your neighbor to try updating their fix with the feedback from Travis...
Hint: it may be late in the day, but keep being nice!
py.test has a feature called test parametrization that can be extremely useful for writing easier-to-understand tests. The key idea is that you can use one simple test function, but with multiple inputs, and break that out into separate tests. At first glance this might appear similar to just one test where you interate over lots of inputs, but it's actually much more useful because it doesn't stop at the first failure. Rather it will run all the inputs ever time, helpinf you debug subtle problems where only certain inputs fail.
For more info and how to actually use the feature, see the py.test docs on the subject. In this challenge problem, try adapting the Problem 2 cases to use this feature. 2c and 2d are particularly amenable to this approach.
Test-driven development is a radically different approach to designing code from what we're generally used to. In test-driven design, you write the tests first. That is, you write how you expect your code to behave before writing the code.
For this problem, try experimenting with test-driven desgin. Choose a problem (ideally from your science interests) where you know some clear cases that you could write tests for. Write the full testing suite (using the techniques you developed above). Then run the tests to ensure all the new ones are failing due to lack of implementation, and then write the new code. A few ideas are given below, but, again, for a real challenge try to come up with your own problem.